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Dear  Educator: 

The  four  booklets  in  this  series  discuss  the  reading, 
mathematics,  science  and  social  studies  results  of  the  1988 
Massachusetts  Educational  Assessment  Program.   They 
represent  one  of  the  many  efforts  of  the  Department  of 
Education  to  help  schools  carry  out  their  educational 
mission  more  effectively.   In  this  case,  they  provide  models 
for  student  evaluation  within  the  classroom,  as  well  as 
describing  students'  progress  in  understanding. 

The  title  of  this  series,  On  Their  Own,  suggests  an 
important  aim  of  education:  the  ability  of  students  to  act 
as  independent,  rational  thinkers.   The  questions  described 
in  these  booklets  demand  that  ability.   They  demand  active 
intelligence  as  students  are  required  to  relate  what  they 
know  to  new  and  challenging  situations. 

In  addition  to  describing  students'  understanding,  these 
booklets  carry  a  message  about  the  evaluation  that  goes  on 
in  the  classrooms.  The  message  is  that  the  short  objective 
tests  of  facts  or  procedures,  standard  fare  in  most 
classrooms,  are  too  slight  a  vehicle  to  convey  the  true 
purpose  of  evaluation. 

In  the  first  place,  effective  student  evaluation  is  an 
important  component  of  effective  teaching.   Research  has 
described  the  complex  thinking  that  underlies  students' 
errors  and  misconceptions.  Unless  teachers  take  the  time  to 
discover  for  themselves  how  students  understand  a  subject, 
they  will  be  unable  to  adjust  their  teaching  in  appropriate 
ways.  This  kind  of  evaluation,  involving  student  discussion 
and  explanation,   should  be  a  continuous  and  constant  part 
of  every  classroom. 


Secondly,  evaluation  can,  and  does,  affect  students' 
learning.   Not  only  does  it  signal  for  the  student  the 
content  areas  that  teachers  consider  important,  it  gives  a 
message  about  the  kind  of  thinking  that  is  considered 
valuable.   When  testing  is  limited  to  short  objective 
questions,  requiring  a  single  answer,  the  message  given  is 
that  facts  are  what  really  count.   When  questions  encourage 
students  to  think,  to  grapple  with  the  material  and  to 
consolidate  different  aspects  of  learning,  the  message  is 
much  different.  Such  questions  indicate  to  students  that  it 
is  the  quality  of  thought  that  is  important,  not  the 
correctness  of  the  answer  itself.   The  possibility  of 
different  answers  opens  the  door  for  discussion, 
argumentation,  and  intellectual  excitement  in  our  schools. 
This  is  the  message  that  we  want  to  convey  to  our  students. 

We  hope  that  you  will  study  the  material  included  in  this 
series  and  incorporate  the  ideas  presented  in  your  own 
classrooms. 


Sincerely  yours, 


Harold  Raync 

Commissioner  of  Education 
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Critics  claim  that  there  are  many  problems  with  current  assessment 
procedures.  Commonly  used  tests  reflect  a  fragmented  view  of  mathe- 
matics rather  than  the  view  of  mathematics  as  an  integrated  whole  and  they 
predominantly  consist  of  items  requiring  factual  knowledge  or  procedural 
skills  rather  than  items  that  assess  students'  abilities  to  think  critically,  to 
reason,  to  solve  problems,  to  interpret,  and  to  apply  ideas  in  creative  ways. 
With  their  focus  on  solutions  rather  than  on  strategies  or  processes,  the 
current  tests  reinforce  in  students,  teachers,  and  the  public  the  narrow  image 
of  mathematics  as  a  subject  with  unique  correct  answers. 

Critics  of  current  mathematics  assessment  procedures  are  consistent  in  what 
they  proposed  to  replace  the  current  procedures.  It  is  essential  that  testing 
be  reorganized  to  parallel  the  new  goals  of  the  curriculum:  the  development 
of  mathematical  understanding,  the  interpretation  of  mathematical  events, 
and  the  application  of  mathematical  procedures.  Because  the  emphasis 
should  be  not  only  on  solutions  but  also  on  processes  and  strategies,  tests 
should  include  open-ended  or  free-response  items,  that  is,  items  for  which 
the  examinee  has  to  create  a  response  rather  than  select  a  response  from  a 
list. 

This  report  presents  information  about  Massachusetts  initial  use  of  open- 
ended  questions  in  their  biennial  assessment  of  mathematics  at  grades  four, 
eight,  and  twelve.  The  summary  of  results  provided  here  make  the  impor- 
tance of  including  such  items  in  any  assessment  apparent. 
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In  the  spring  of  1988,  the  Massachusetts  Department  of  Education  ad- 
ministered its  second  biennial  assessment.  It  tested  all  eligible  fourth, 
eighth,  and  twelfth  grade  students  in  four  content  areas  — reading,  mathe- 
matics, science,  and  social  studies.  Although  the  large  majority  of  the  over 
3000  items  were  given  in  multiple-choice  format,  some  of  the  items  were 
open-ended,  requiring  students  to  answer  in  written  form.  These  open-ended 
questions  appeared  in  one  form  of  the  tests  at  each  grade  level.  Consequently, 
one-twelfth  of  the  fourth  grade  students,  one-sixteenth  of  the  eighth  grade 
students,  and  one-twentieth  of  the  twelfth  grade  students  received  a  test  form 
that  contained  some  open-ended  questions.  However,  they  did  not  all  receive 
the  same  questions.  The  ten  or  so  questions  in  each  subject  area  were 
distributed  in  such  as  way  that,  while  we  were  able  to  receive  a  sufficiently 
large  number  of  responses  to  each  question  to  report  reliably,  we  were  not 
able  to  cover  the  many  types  of  thinking  that  each  subject  required. 

Members  of  our  Advisory  Committees,  composed  of  teachers  throughout  the 
state,  reviewed  and  categorized  all  the  answers  obtained.  Their  comments 
and  instructional  suggestions  are  reflected  in  this  report. 

There  are  three  reasons  for  our  decision  to  include  open-ended  questions  in 
our  assessment  of  student  performance.  The  first  is  the  value  of  the  informa- 
tion obtained.  While  multiple-choice  items  are  efficient,  easy  to  score,  and 
objective,  they  are  a  weak  measure  of  how  students  actually  think.  Nor  can 
they  measure  students'  ability  to  generate  solutions  or  students'  approach  to 
those  ill-structured  problems  that  are  most  familiar  in  everyday  life.  Includ- 
ing these  types  of  questions  on  the  assessment  results  in  a  more  valid  estima- 
tion of  student  achievement  than  we  would  have  obtained  had  we  limited 
assessment  to  multiple-choice  items. 

Our  second  purpose  in  including  these  open-ended  questions  was  our  belief 
in  their  intrinsic  value:  they  reflect  the  kind  of  thinking  that  education  is  all 
about.  Too  often  educators  pay  lip  service  to  the  need  for  active  learning  but 
teach  and  test  students  in  ways  that  demand  passivity.  By  their  actions, 
schools  say  to  students,  "We  are  not  interested  in  your  response;  we  are  only 
interested  in  the  correct  response."  This  report  of  the  open-ended  testing 
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shows  how  students  respond  when  they  are  challenged  to  conceptualize  a 
problem  as  well  as  deal  with  it. 

Finally,  we  hope  that  such  testing  will  act  as  a  model  for  classroom  testing. 
We  have  given  the  questions  themselves,  as  well  as  the  state-wide  results,  in 
order  for  teachers  to  try  them  out  in  their  own  classes  and,  if  they  desire,  to 
compare  the  results  they  obtain  with  the  state  norm.  We  wish  to  show  that 
this  type  of  testing  yields  important  information  about  students'  under- 
standing of  concepts  and  procedures,  their  ability  to  apply  their  learning  to 
new  situations,  and  their  need  for  further  instruction. 

Although  the  results  in  each  subject  area  are  treated  separately,  the  underly- 
ing thought  processes  which  we  report  on  are  similar.  They  reflect  an  ap- 
proach to  thinking  that  stresses  engagement  and  critical  evaluation.  Beyond 
this,  however,  we  look  at  how  students  function  in  various  learning  contexts 
which  require  the  understanding  of  specific  concepts. 


A  New  Vision  of  School  Mathematics 

During  the  last  two  years,  it  has  been  generally  conceded  that  there  is 
something  wrong  about  the  way  mathematics  is  taught  and  learned  in 
this  country.  When  the  National  Assessment  for  Educational  Progress  issued 
its  Mathematics  Report  Card,  students  received  poor  marks  across  the 
board.  It  was  reported  that  virtually  no  9-year-olds,  and  only  16  percent  of 
13-year-olds  and  51  percent  of  17-year-olds  could  deal  with  moderately 
complex  mathematical  procedures  and  reasoning,  e.g.,  finding  the  area  of  a 
rectangle.  Only  6  percent  of  17  year  olds  could  use  more  complex  reasoning 
or  algebra  to  solve  problems. 

Later  in  the  year,  an  international  study  of  mathematics  and  science  achieve- 
ment among  13-year-olds  confirmed  these  results.  United  States  students 
obtained  the  lowest  mathematics  score  of  any  participating  group.  In  one 
category,  the  use  of  intermediate-level  skills  to  solve  two-step  problems,  there 
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was  a  38  percent  difference  between  the  achievement  level  of  13-year-olds  in 
this  country  and  in  Korea,  the  high  scoring  country.  Only  9  percent  of  our 
students  showed  an  understanding  of  measurement  and  geometry  concepts, 
in  contrast  to  40  percent  of  Korean  students. 

The  following  year,  the  National  Research  Council  and  the  National  Council 
of  Teachers  of  Mathematics  (NCTM)  suggested  remedies.  They  looked  to 
the  future  with  two  publications  that  challenged  the  way  that  mathematics  is 
conceived  of  and  taught  in  the  schools  and  gave  their  vision  for  a  more 
meaningful  mathematics  education. 

A  key  point  made  by  both  organizations  is  that  mathematics  is  and  always  has 
been  an  evolving  intellectual  area  that  connects  the  theoretical  and  practical 
world.  From  the  time  that  Kepler  anticipated  the  concept  of  integral  calculus 
by  trying  to  calculate  the  best  shape  for  Austrian  wine  casks  to  the  present 
concern  of  mathematicians  in  trying  to  define  the  structural  pattern  of  random 
events  in  natural  phenomena,  mathematics  has  consisted  of  both  abstraction 
and  application,  constantly  crossing  between  the  worlds  of  ideas  and  prac- 
ticality. This  is  something  that  few  students  realize  as  they  march,  and  often 
stumble,  through  the  grind  of  arithmetic  calculations. 

The  National  Council  of  Teachers  of  Mathematics  believes  that  appreciation 
of  what  mathematics  really  is  should  permeate  its  teaching.  One's  first 
impression  when  reading  through  the  NCTM's  Curriculum  and  Evaluation 
Standards  is  the  variety  of  mathematical  topics  that  are  covered.  One's 
second  impression  is  that  many  of  the  same  ideas  and  concepts  reappear  in 
different  forms.  Mathematics  is  a  connected  body  of  thought.  It  is  connected, 
not  only  with  other  subjects  — science,  social  studies,  as  well  as  art  and 
music  — but  as  a  subject  within  itself.  An  underlying  structure  unites  the 
various  mathematical  topics,  while  the  same  phenomena  can  be  represented 
visually,  algebraically,  statistically.  An  appreciation  of  that  basic  connected- 
ness (structure)  is  what  the  authors  of  the  Standards  call  "mathematical 
power." 

How  students  can  attain  that  power  is  what  the  Standards  are  about.  They 
describe  the  basic  concepts  and  skills  that  are  appropriately  taught 
throughout  the  school  years,  justify  their  mathematical  importance,  and  give 


National  Research  Council,  Everybody  Counts.  Washington,  D.C.:  National  Academy 
Press,  1989.  National  Council  of  Teachers  of  Mathematics,  Curriculum  and  Evaluation 
Standards  for  School  Mathematics.  Reston,  Va.:  NCTM,  1989. 


On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


guidance  to  teachers  on  how  they  should  be  presented.  The  vision  that  they 
describe  emphasizes  the  logical  consistency  of  mathematics. 

The  ideas  that  underlie  these  mathematical  goals  are  very  similar  to  those 
that  underlie  the  reform  in  other  subjects  — the  importance  of  prior 
knowledge  and  preconceptions,those  ideas  and  concepts  that  students  bring 
to  the  classroom  experience;  the  need  for  active  learning,  whereby  students 
make  ideas  their  own;  the  explicit  reference  to  real-life  problems;  and  the 
importance  of  reasoning  and  communication.  However,  in  contrast  to  other 
subject  areas,  this  vision  could  be  seen  as  revolutionary.  It  is  certainly  not  the 
picture  of  mathematics  as  it  is  taught  in  the  majority  of  schools  today. 

We  use  a  single  incident  to  illustrate  our  point.  During  our  pilot  testing  of 
the  open-ended  items,  a  twelfth  grade  student  looked  at  them  with  a  sense  of 
outrage  and  muttered,  "These  aren't  math!"  Putting  his  head  on  the  desk,  he 
refused  to  go  on.  This  student  was  clearly  insulted  by  being  confronted  with 
something  that  was  unfamiliar,  requiring  him  to  apply  what  he  had  studied  to 
a  context  that  was  different  from  his  daily  routine  of  textbook  problems.  He 
was  unprepared  even  to  try.  And  yet,  few  of  the  problems  contained  in  the 
open-ended  form  of  the  assessment  were  "difficult"  problems.  Many  could 
be  answered  by  common  sense  or  by  applying  some  fundamental  mathemati- 
cal concepts.  Their  difficulty  lay,  not  in  the  knowledge  that  they  required,  but 
in  their  unfamiliarity  and  their  demand  for  understanding  rather  than  proce- 
dure per  se.  In  this  regard,  they  are  similar  to  the  demands  of  those  so-called 
mathematical  problems  of  everyday  life.  Mainly,  they  demand  a  flexibility, 
an  ability  to  see  the  underlying  mathematics  and  to  relate  learned  knowledge 
to  a  new  situation.  Unless  students  are  exposed  to  these  kinds  of  problems 
they  will  continue  to  see  mathematics  as  something  useless,  that  is  carried  out 
in  a  40-minute  class,  with  no  relevance  to  the  real  world  outside  the  school 
boundaries. 

The  set  of  open-ended  questions  which  we  present  in  this  booklet  include 
some  fundamental  ideas  of  mathematics.  They  are  not  designed  as  models 
for  the  kinds  of  problems  that  required  extended  investigation.  Although 
extended  problems  should  be  part  of  everyone's  school  experience,  they  are 
not  appropriate  for  the  limited  time  period  allowed  for  the  assessment.  The 
problems  presented  here  are  intended  as  models  for  teaching,  as  well  as  for 
evaluation.  These  are  classroom  exercises  that  are  appropriate  for  all  stu- 
dents. In  that  context  they  serve  a  double  role.  They  allow  students  to  begin 
to  think  and  talk  about  mathematics,  to  share  their  ideas  and,  perhaps,  to 
become  excited.  Each  could  be  a  starting  point  for  class  or  individual  discus- 
sion. They  also  allow  teachers  to  gain  insight  into  how  their  students  think 
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about  mathematics.  When  practical,  each  of  the  questions  is  also  accom- 
panied in  the  Appendix  by  a  chart  giving  the  percentages  of  responses 
throughout  the  sample.  These  are  presented  for  the  convenience  of  teachers 
who  might  wish  to  compare  the  performance  of  their  own  students  with  those 
across  the  state.  Mathematics  teachers  should  also  consult  a  companion 
publication,  On  Their  Own:  Student  Response  to  Open-Ended  Tests  in 
Science,  for  problems  involving  data  reduction  and  analysis. 
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The  open-ended  items  that  were  presented  in  the  assessment  covered 
three  major  areas  in  school  mathematics:  Patterns  and  Relationships, 
Geometry  and  Measurement,  Numerical  and  Statistical  Concepts. 


Patterns  and  Relationships 

Experience  in  recognizing  patterns  and  relationships  is  not  an  "enrichment 
activity."  To  a  much  greater  degree  than  arithmetic  calculations,  it  repre- 
sents the  real  work  of  mathematics,  with  two  important  benefits  for  students. 
One  is  affective.  If  we  want  students  to  appreciate  mathematics,  to  see  it  as 
a  reasonable  activity  that  they  can  control  for  their  own  purposes,  then  we 
cannot  just  give  them  a  set  of  rules  to  follow.  They  must  be  able  to  act  like 
mathematicians,  discovering  for  themselves  the  numerical  and  geometric 
structures  that  are  the  essence  of  mathematics.  Discovering  patterns  is  not 
only  satisfying  in  itself,  it  gives  students  a  sense  of  confidence  in  the 
reasonableness  of  mathematics. 

Secondly,  work  with  numerical  and  graphical  patterns  lay  the  groundwork  for 
understanding  functions  and  the  relationship  between  variables.  Without  an 
intuitive  feeling  for  what  the  symbols  of  algebra  represent,  algebraic 
manipulations  remain  just  that  —  manipulations. 
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Numerical  Patterns  and  Relationships 
Grade  Four 

Fourth  grade  students'  ability  to  recognize  structure  in  numerical  patterns 
was  tested  in  the  following  question,  which  was  also  asked  on  the  multiple- 
choice  section  of  the  test,  without  the  requirement  that  students  explain  their 
method. 

When  numbers  are  put  in  a  number  machine,  different  numbers 
come  out. 

If  a  3  goes  in,  a  5  comes  out. 
If  a  4  goes  in,  a  7  comes  out. 
If  a  5  goes  in,  a  9  comes  out. 

If  an  8  goes  in,  what  number  comes  out? 

Explain  in  words  how  you  came  up  with  your  answer. 


The  table  below  shows  how  the  responses  to  the  two  versions  of  the  question 
differed. 


Number 

Percentage  of  responses: 

produced 

open 

-ended 

multiple-choice 

15    * 

39 

28 

12 

15 

50 

14 

5 

18 

23 

- 

4 

13 

19 

- 

11 

4 

- 

other 

16 

- 

blank 

2 

0 

The  students  who  were  given  the  open-ended  version  of  the  question  were 
more  successful  than  those  who  answered  the  multiple-choice  version.  Per- 
haps the  requirement  that  they  explain  their  method  prompted  them  to  think 
more  carefully  about  the  relationship.  In  any  case,  their  explanations  gave  us 
some  insight  into  how  students  arrived  at  incorrect  responses  on  both  versions 
of  the  question.  For  example,  half  the  students  incorrectly  chose  12  as  the 
answer  when  presented  with  pre-selected  options,  while  15  percent  gave  it  in 
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response  to  the  open-ended.  From  the  explanations  given,  it  was  apparent 
that  these  students  focused  exclusively  on  the  final  pair  of  numbers  (5  and  9), 
without  considering  all  the  sets  of  numbers.  Noting  a  difference  of  four,  they 
generalized:  I  got  12  because  8  plus  4  is  12  or  If  you  put  5  in,  a  9  conies  out. 
So  if  you  put  8  in  a  12  must  come  out  because  of  6..7..8..9  and  9..10..1 1..12. 
12  is  the  last  answer,  so  it  must  come  out. 

Although  14  was  a  popular  choice  when  presented  as  an  option  in  multiple- 
choice,  few  students  volunteered  it  when  they  had  no  options  to  choose  from. 
Instead,  many  (20  percent)  gave  the  answer  as  13.  In  general,  this  seemed  to 
be  the  result  of  carelessness  rather  than  a  misunderstanding  of  the  relation- 
ship between  the  two  sets  of  numbers.  For  example,  many  students  recog- 
nized the  functional  relationship  but  ignored  or  miscounted  the  missing 
elements  in  the  series  (6  and  7).  A  typical  explanation  for  this  type  of  error 
was:  The  first  one  adds  2  numbers,  and  the  second  one  adds  3,  and  the  third 
one  adds  4,  and  so  on.  Number  8,  you  have  to  add  5  to  it  and  it's  a  13. 

A  large  number  (16  percent)  recognized  only  very  general  relationships  or 
guessed:  Because  all  the  numbers  that  go  in  are  even,  and  when  they  come 
back  out  they  are  odd. 

As  well  as  lending  insight  into  why  students  chose  the  incorrect  answer,  the 
requirement  for  explanation  also  gave  an  indication  of  students'  reasoning 
when  they  chose  the  correct  solution.  It  cast  doubt  on  how  well  some  students 
understood  the  relationship  involved.  For  example,  of  those  who  gave  the 
correct  answer  of  15,  only  a  third  were  able  to  spot  some  kind  of  functional 
relationship  which  they  were  able  to  express  in  words. 

You  double  the  number  you  have  and  subtract  one. 

I  looked  at  all  the  figures  and  subtracted  the  number  that  went  in 
from  the  number  that  came  out.  I  found  out  that  the  answer  was  1 
less  than  the  number  that  went  in.  Then  I  took  1  off  of  8,  which 
was  7.  Then  I  said  8  +  7  =  15. 


Another  third  focused  only  on  the  second  set  of  numbers,  treating  it  as  a 
sequence:  Each  number  that  comes  out  is  2  away  from  each  other.  A  final 
third  who  gave  the  correct  answer  could  not  explain  their  reasoning:  I  found 
out  the  pattern  and  completed  it. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Grade  Eight 

At  eighth  grade,  we  asked  students  for  an  algebraic  as  well  as  a  verbal 
explanation  of  the  relationship  between  two  sets  of  variables. 

Students  were  given  the  following  question: 


Sample: 


The  missing  number 


What  you  do  to  x  to  get  y: 


The  equation: 


X 

y 

0 

1 

1 

2 

3 

□ 

4 

5 

6 

7 

Now  try  this  one: 


X 

y 

0 

1 

2 

5 

4 

□ 

7 

15 

9 

19 

The  missing  number 


What  you  do  to  x  to  get  y: 


The  equation 


As  in  the  previous  question,  the  correctness  of  the  solution  gave  only  slight 
indication  that  students  understood  the  relationship  involved.  (Results  are 
given  in  Appendix.) 

Although  approximately  half  the  students  gave  the  correct  answer  of  9,  their 
subsequent  explanations  showed  that  far  fewer  recognized  the  function  that 
existed  between  the  two  sets  of  numbers.  A  substantial  number  (the  10 
percent  who  answered  "y  =  x  +  5")  focused  on  the  specific  case.  They  may 
have  been  misled  by  the  example  given  (y  =  x  + 1),  in  which  the  equation  both 
described  the  missing  number  and  the  set  of  relationships.  However,  they 
most  certainly  did  not  understand  the  question  as  asking  for  the  function  that 
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relates  the  two  sets  of  numbers.  Another  13  percent,  while  giving  the  correct 
number,  were  unable  to  supply  an  adequate  explanation  or  an  equation. 

The  remaining  half  of  the  students  were  obviously  baffled  by  the  task.  There 
was  a  great  variety  of  answers.  Few  of  these  students  were  able  to  explain 
how  they  arrived  at  their  solutions,  with  some  students  merely  copying  the 
equation  given  in  the  example. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Grades  Eight  and  Twelve 

A  third  numerical  relationship  problem  was  presented  to  students  at  Grades 
8  and  12. 


Sample: 


The  missing  number 
The  equation: 


X 

y 

0 

1 

1 

2 

3 

□ 

4 

5 

6 

7 

Now  try  this  one: 


X 

y 

1 

6 

2 

8 

3 

10 

4 

12 

10 

□ 

The  missing  number. 


Explain  how  you  found  the  missing  number 


The  equation 


This  problem  presented  more  difficulty  to  students  for  two  reasons.  Whereas 
the  previous  eighth  grade  problem  alerted  students  to  a  relationship  between 
x  and  y  by  asking  them  to  explain  what  they  did  to  x  to  get  y,  this  question  gave 
no  hint  that  the  two  were  related.  It  required  students  merely  to  "explain." 
Secondly,  the  number  for  each  variable  was  presented  in  order  (e.g., 
l,2,3,4..and  6,8,10,12..).  This  prompted  students  to  look  at  them  as  two 
separate  sequences  with  no  relationship  to  one  another.  (See  Appendix.) 

Fifty-two  percent  of  eighth  graders  and  35  percent  of  twelfth  graders  gave  an 
incorrect  solution  and  10  percent  at  both  grades  did  not  attempt  the  question. 
Most  of  the  incorrect  responses  focused  on  the  columns,  treating  the  numbers 
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as  sequences.  When  asked  to  supply  the  equation,  these  students  merely 
ignored  the  functional  relationship  between  the  sets  and  wrote  a  number 
sentence  relating  the  specific  pair.  The  pattern  is  x  adds  one  each  time,  y 
adds  2,  wrote  one  Grade  8  student.  (Equation  given:  y  =  x  +  14.) 

Thirty-eight  percent  of  eighth  graders  and  fifty-five  percent  of  twelfth  graders 
supplied  the  correct  missing  number  (24).  However,  although  there  was  an 
increase  in  success  rate  between  the  two  grades,  the  percentage  of  students 
who  were  able  to  explain  the  operation  or  supply  an  equation  is  disappointing. 
Only  21  percent  of  eighth  graders  and  34  percent  of  twelfth  graders  were  able 
to  explain  their  reasons  for  choosing  the  correct  number.  Only  13  percent  of 
eighth  graders  were  able  to  give  the  equation  which  describes  the  relation- 
ship. Although  this  number  rose  to  28  percent  at  the  twelfth  grade  level,  it  is 
disappointing,  given  the  number  of  twelfth  graders  who  have  taken  basic 
algebra. 


a 

b 

1 

6 

2 

8 

3 

10 

4 

12 

Which  rule  fits  this  table? 

A.  b  =  (3  x  a)  +  3 

B.  b  =  (2  x  a)  •«■  3 

C.  b  =  (3  x  a)  +  2 

D.  b  =  (2  x  a)  +  4 


Percentage  of  correct  responses: 
Gr8  Gr12 


18% 

12% 

A. 

b  =  (3  x  a)  +  3 

20 

10 

B. 

b  =  (2  x  a)  +  3 

22 

16 

C. 

b  =  (3  x  a)  +  2 

38 

61      * 

D. 

b  =  (2  x  a)  +  4 

Although  students  were  more  successful  when  given  a  set  of  equations  to 
choose  from  (see  multiple-choice  above),  the  results  in  response  to  the 
open-ended  version  suggest  the  difficulties  that  students  have  in  generating 
equations  or  using  algebraic  notation  as  a  shorthand  for  description. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Graphical  Relationships 


The  recognition  of  pattern  and  relationship  is  not  confined  to  numbers.  It 
underlies  the  understanding  of  graphical  material  as  well. 

Although  students  perform  relatively  well  when  asked  to  read  data  from  bar 
graphs,  they  are  not  as  successful  at  interpreting  graphical  display.  Response 
to  the  science  open-ended  tasks  indicated  that  students  experience  great 
difficulty  in  depicting  relationships  between  variables.  (See  On  Their  Own: 
Student  Response  to  Open-Ended  Questions  in  Science.)  The  questions 
below  do  not  involve  the  mechanics  of  graphing  perse  (scales,  intervals,  etc.). 
Instead  they  addressed  some  basic  concepts  by  asking,  do  students  understand 
graphs  as  the  representation  of  a  relationship  between  variables? 


Grade  Four 


1 .    A  school  has  twelve  grades  in  it.  A  student  computes  the  average  height  of  the  students 
in  each  grade  and  makes  a  graph  showing  the  results.  Circle  the  graph  below  that 
would  look  most  like  the  student's  graph. 


Average  Height  ot 

Students  in  Each 

Grade  (inches) 


Average  Height  ol 

Students  in  Each 

Grade  (inches) 


Average  Height  of 

Students  in  Each 

Grade  (inches) 


t!)ISt!t  8  101112 

Grade 


1  2  3  4  5  8  7  e  6  10U12 

Grade 


123456719  101112 

Grade 


2.     Explain  why  you  chose  the  graph  you  did  in  question  1 . 


Most  fourth  graders  showed  a  good  understanding  of  what  the  graphs  were 
intended  to  convey.  Eighty  percent  of  the  students  selected  the  first  (correct) 
graph.  Sixty-six  percent  gave  a  convincing  reason.  "As  you  get  older  you  get 
taller,  and  in  the  first  graph  the  students  get  bigger  and  bigger,"  wrote  one 
student. 

"The  people  in  the  second  graph  are  all  mixed  up,"  another  responded.  "I 
don't  think  the  first  graders  are  taller  than  9th  and  10th  graders.  In  the  third 
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graph,  it's  mixed  up  too.  1st  graders  are  not  the  same  height  as  12th  graders 
and  2nd  graders  are  taller  than  10th  graders?  I  don't  think  so." 

Fifteen  percent  chose  graph  2.  Approximately  half  of  those  children 
misunderstood  the  meaning  of  the  horizontal  axis,  believing  that  each  child 
was  represented  by  a  number:  "Because  it's  not  in  order.  Not  all  12th  graders 
are  tall." 

Others  based  their  decision  on  its  general  appearance:  "The  others  look  too 
much  like  a  made-up  school  because  they  have  patterns."  The  4  percent  of 
the  students  who  chose  the  third  graph  were  typically  unable  to  explain  their 
choice. 

Judging  by  their  responses  to  this  question,  it  seems  that  the  large  majority  of 
fourth  graders  were  able  to  transfer  their  experience  with  bar  charts  of 
frequencies  or  amounts  to  a  more  sophisticated  context  (i.e.,  average  height), 
as  well  as  recognizing  the  relationship  between  age  and  height. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Grades  Eight  and  Twelve 

A  parallel  question  at  the  eighth  and  twelfth  grades  asked  students  to  inter- 
pret data  presented  in  a  line  graph.  The  results  show:  1)  a  general  increase 
in  success  with  experience;  2)  a  gender  difference  that  is  associated  with  age; 
and  3)  the  influence  of  preconception  on  the  interpretation  of  graphs. 


1 .    A  researcher  asked  many  students  two  questions:  "What  was  your  grade  on  your  last 
math  exam?"  and  "How  many  hours  per  night  (to  the  nearest  half  hour)  did  you  usually 
spend  on  math  homework?"  The  researcher  than  sorted  students  into  groups  accord- 
ing to  how  much  time  they  spend  on  homework.  Finally,  the  researcher  computed  an 
average  math  grade  for  each  of  these  groups,  and  plotted  the  averages  in  the  graph 
below. 


Grade    in    Math 
(numerical    grade) 


Time    Spent   on    Math 

Homework    per    Evening 

(average) 


A  quarter  of  the  eighth  grade  students  were  able  to  supply  a  complete 
explanation  of  both  negative  and  positive  relationship.  Another  quarter, 
undoubtedly  influenced  by  their  preconceptions  about  the  effect  of  studying, 
could  only  account  for  the  positive  relationship.  As  one  student  wrote:  "It 
goes  from  the  low  grades  to  the  high  grades  back  to  the  low  grades.  The  low 
grades  are  also  the  students  who  spend  less  time  on  their  homework." 

The  other  half  of  the  students  were  unable  to  interpret  the  graph.  Some  were 
confused  by  the  lack  of  scale:  "This  graph  does  not  have  enough  information 
such  as  how  many  hours?  Does  it  increase  by  5,  1,  1/2?  Therefore  I  cannot 
answer  the  question  correctly."  Others  focused  on  only  one  or  the  other 
dimension  without  understanding  that  the  graph  described  a  relationship 
between  variables:  "Some  scores  started  low  and  went  up  and  then  went  down 
again."  and  "The  time  spent  goes  up  and  then  goes  down."  As  in  the  case  of 
the  fourth  graders,  some  students  believed  that  each  interval  represented  a 
discrete  score.  "Because  different  people  had  different  answers." 
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At  the  twelfth  grade  level,  more  students  were  able  to  account  for  the 
curvilinear  shape  of  the  slope.  Nevertheless,  the  incorrect  responses  fol- 
lowed the  pattern  of  the  eighth  graders.  (See  Table  below  and  Appendix.) 

Girls  generally  experienced  more  difficulty  with  this  item  than  boys,  and  this 
gender  difference  increased  with  age.  Whereas  6  percent  more  boys  than  girls 
correctly  interpreted  the  graph  at  grade  eight,  there  was  a  19  percent  dif- 
ference at  grade  twelve.  Boys  who  understood  the  graph  gave  a  complete 
explanation,  whereas  girls  tended  to  explain  only  the  positive  relationship 
between  grades  and  homework  time  or  did  not  address  the  relationship  at  all. 


Graph  of  Study  Time  and  Grades 


Gr  8  Gr  12 

M      F  M      F 


Complete  explanation  (accounts  for  27  21  49  30 
both  positive  and  negative  slopes) 

Partial  explanation  (accounts  for  31  24  25  34 
positive  or  negative  slope) 

Does  not  address  relationship  35  46  13  27 

Blank/I  don't  know  7  9  11  8 


Prior  knowledge  or  preconceptions  seemed  to  influence  students'  interpreta- 
tion of  the  graph.  Whereas  younger  students  found  it  difficult  to  believe  that 
homework  and  grades  were  not  positively  related  (and,  consequently,  tended 
to  ignore  the  negative  relationship  that  was  portrayed),  older  students  talked 
about  fatigue  and  ability.  It  may  be  that  experience  with  the  content  ac- 
counted for  the  increased  success  at  grade  12. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Grades  Eight  and  Twelve 

In  another  set  of  items,  students  were  asked  to  construct  their  own  visual 
relationship.  Because  we  were  interested  in  their  understanding  of  functional 
relationship,  we  did  not  ask  them  to  consider  scales,  the  ordering  of  values, 
and  the  identification  of  coordinates.  The  results  of  the  1986  open-ended 
question  requiring  them  to  draw  a  graph  suggested  that  students  have  great 
difficulty  with  these  aspects  of  visual  representation. 


1 .    Two  of  the  three  graphs  below  have  not  been  completed.  Draw  lines  in  the  last  two  so 
that  the  graphs  look  the  way  they  would  if  you  had  real  numbers  to  use.  (Hint:  Look  at 
the  labels  along  the  axes  first.  Then  pretend  there  are  numbers  along  them.  Think 
about  how  the  two  quantities  are  related.  Use  the  first  graph  as  an  example.) 


Weight 
(pounds) 


Time    Needed 

to    Run    Mile 

(minutes) 


Auto    Resale 
Value 


Height   (inches) 


Age    (years) 


Age   ol   Auto   (years) 


2.     Explain  why  you  drew  the  graphs  the  way  you  did. 


Students  responses  to  the  time/age  graph  differed  according  to  their  perspec- 
tive. Consequently,  graphs  were  judged  by  the  extent  to  which  they  reflected 
the  explanation  given.  For  example,  when  students  compared  themselves 
with  much  older  people  and  stated:  "As  you  get  older,  the  more  time  you  need 
to  run  a  mile,"  a  straight  linear  relationship  was  considered  correct.  On  the 
other  hand,  when  a  student  wrote  of  the  changing  pattern  from  childhood  to 
old  age,  a  curvilinear  slope  was  required. 

Even  accounting  for  the  flexibility  in  scoring,  this  proved  to  be  a  difficult  task 
for  all  students.  (See  Appendix  for  examples.)  Only  29  percent  of  eighth 
graders  and  34  percent  of  twelfth  graders  were  able  to  draw  slopes  that 
reflected  their  statements. 

Many  of  the  incorrect  responses  were  visual  in  character,  particularly  among 
twelfth  grade  girls.  These  students  appeared  to  think  in  terms  of  an  increase 
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in  speed  rather  than  a  decrease  in  time.  Their  explanations  described  age  as 
related  to  "running  fast"  and,  consequently,  they  tended  to  draw  positive 
slopes.  Approximately  3 1  percent  of  twelfth  grade  girls  fell  into  this  category, 
in  contrast  to  11  percent  of  the  boys.  When  boys  drew  an  incorrect  slope, 
their  explanation  showed  a  more  general  lack  of  understanding.  The  example 
below  illustrates  a  typical  response.  It  also  reflects  the  tendency  among 
students  to  start  the  slope  at  the  intercept  of  the  axes,  regardless  of  how  the 
relationship  is  described. 


WeioM 
(pounds) 


Time    Needed 

to    Run    Mile 

(minutes) 


Auto    Resale 
Value 


Height    (inches) 


Age    (years) 


Ago    ol    Auto    (years) 


In  the  second  graph,  very  young  children,  especially  babies,  are 
not  very  quick  runners  usually.  When  people  start  to  get  older, 
then  into  their  prime,  their  legs  get  more  mature  to  run  faster.  As 
they  get  older  they  get  weaker  bones  and  muscles,  so  they  can't 
run  as  fast. 

A  new  car  can  sell  for  the  most  because  it  is  unused  and  in  perfect 
condition.  As  the  car  gets  older  it  gets  more  worn  down,  goes 
more  miles,  gets  more  dents  and  scratches,  etc.  so  the  price 
decreases. 


The  third  graph,  which  concerned  the  relationship  between  a  car's  age  and 
its  value,  was  perhaps  of  greater  interest  and  familiarity  to  the  students.  It 
had  a  much  higher  success  rate,  and  the  differences  between  boys  and  girls 
were  also  less  apparent.  This  may  be  due  to  the  fact  that,  in  this  item,  the 
verbal  and  visual  information  reinforce  each  other,  as  a  decline  in  values 
suggests  a  descending  slope.  However,  some  students  recognized  that  the 
slope  can  be  curvilinear.  The  following  example  illustrates  the  value  of 
requiring  an  explanation  to  such  items;  otherwise,  who  would  know  that  the 
student  was  referring  to  antique  cars  in  her  graph? 

2nd  graph  I  did -The  price  of  cars  go  down,  until  they're  antique. 
The  prices  then  rise. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Geometry  and  Measurement 

Geometry:  Plane  Figures 

Students'  responses  to  multiple-choice  questions  concerning  geometry  have 
been  described  in  detail  in  a  separate  publication,  Moving  Geometry  from 
the  Back  of  the  Book  (Department  of  Education,  1987).  In  the  open-ended 
tests,  we  examined  fourth  graders'  conception  of  a  simple  plane  figure  by 
asking  them  to  describe  a  rectangle. 


Describe  a  rectangle  in  words  for  a  person  who  does  not  know  what  a 
rectangle  is.  Describe  it  completely  so  that  the  person  will  be  able  to  tell 
if  any  figure  is  or  is  not  a  rectangle. 


While  3  percent  of  the  students  were  able  to  give  a  complete,  defining 
description  of  a  rectangle,  approximately  half  gave  an  adequate  informal 
description.  (See  Appendix  for  complete  results.)  These  informal  descrip- 
tions included  the  notion  of  four  sides  and  pairs  of  equal  sides.  As  expected, 
students'  descriptions  tended  to  stress  the  prototypical  long  rectangle  and 
they  showed  the  usual  tendency  of  differentiating  between  rectangle  and 
square.  Generally,  few  students  mentioned  angles,  although  9  percent  used 
the  word  "corners"  to  describe  right  angles.  A  typical  response  was: 

A  rectangle  is  a  figure  with  four  sides.  Two  sides  are  long  and  two 
sides  are  short.  The  rectangle  is  two  squares  put  together.  A  rec- 
tangle has  four  corners. 


Other  students  referred  to  common  objects  to  help  them  in  their  description, 
such  as,  "It  looks  like  a  stick  of  gum."  However,  many  were  unclear  about  the 
figure.  (See  Appendix.)  For  example,  13  percent  described  a  triangle.  One 
very  graphic  description: 

A  rectangle  is  two  slanted  lines  with  a  straight  line  at  the  bottom 
and  it  is  shaped  like  an  A  but  the  line  in  the  middle  gets  taken 
away  and  put  at  the  bottom  of  the  A. 
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A  Question  of  Answers 


Still  others  referred  to  a  solid  figure,  while  retaining  the  concept  of  length: 

A  rectangle  is  like  a  cube  but  it  is  larger  than  a  cube  and  it  has  6 
sides  and  it  has  no  curved  surfaces,  but  it  does  have  flat  surfaces 
and  it  has  six  vertexes  and  it  is  longer  than  a  cube  too. 

A  rectangle  has  4  sides.  It  is  shaped  like  a  tissue  box.  It  is  not 
shaped  like  a  cube. 


The  fact  that  50  percent  of  the  students  did  not  give  an  adequate  description 
of  a  rectangle  indicates  how  fragile  their  understanding  of  plane  figures  may 
be.  When  asked  to  recognize  prototypical  figures  as  on  the  multiple-choice 
questions,  students  tend  to  perform  well.  For  example,  when  presented  with 
a  set  of  standard  plane  figures,  over  90  percent  of  third  graders  correctly 
identified  rectangles,  squares,  and  triangles.  However,  the  multiple-choice 
question  may  give  us  an  inflated  impression  of  what  students  know.  Descrip- 
tion demands  a  greater  degree  of  understanding.  In  order  to  describe  a  figure, 
students  must  consider  its  properties,  those  attributes  that  make  it  what  it  is. 
This  is  active  knowledge,  which  many  children  seem  to  lack. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Geometry  and  Measurement:  Area 

An  understanding  of  the  relationship  between  geometry  and  measurement 
has  important  practical  uses,  as  well  as  being  crucial  to  success  in  more 
advanced  courses  in  calculus.  The  questions  below  focus  on  the  relationship 
between  geometry  and  measurement,  asking  students  to  express  the  area  of 
a  figure  in  terms  of  a  non-traditional  unit.  Here,  we  were  particularly  inter- 
ested in  students'  ability  to  imagine  how  a  figure  might  be  partitioned  —  a 
necessary  condition  for  calculating  the  area  of  irregular  shapes. 


Grade  Four 


IS    ONE    UNIT    OF    AREA. 


What    is    the    area    of    the    figure    below? 


AREA:. 


units 


The  most  striking  feature  about  the  responses  to  this  item  is  the  effect  of 
partitioning  on  the  accuracy  of  response. 

Over  80  percent  of  the  students  who  partitioned  were  able  to  calculate  the 
correct  answer  (14  units),  while  fewer  than  30  percent  of  those  who  did  not 
partition  were  successful.  A  slightly  disappointing  note  was  the  number  of 
students  who  erased  their  partitions,  presumably  after  they  had  counted  the 
units.  Evidently,  they  believed  that  their  method  might  be  considered  cheat- 
ing or  was  somehow  unacceptable. 
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A  Question  of  Answers 


Fourth  Grade 


Partitioned,  (5.0%) 
Incorrect 


No  Partitioning,  (16.0%) 
Correct 


Partitioned,  Correct  (35.0%) 


Blank/Irrelevant 
(10.0%) 


No  Partitioning,  Incorrect  (34.0%) 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Grades  Eight  and  Twelve 

At  the  eighth  and  twelfth  grade  levels,  the  measuring  unit  was  more  complex, 
although  the  question  was  essentially  the  same. 


IS    ONE    UNIT    OF    AREA. 


What    is    the    area    of    the    figure    below? 


-• •- 


•         •         •         • 


»  « • r- 


•  •  •  • 

AREA: units 


As  a  result  of  either  the  unit  itself  or  a  lack  of  recent  experience,  only  a  quarter 
of  eighth  graders  attempted  to  partition  the  figure.  Eighty  percent  of  the 
eighth  graders  did  not  partition,  with  few  giving  the  correct  answer  of  5  units. 
The  most  popular  answer  was  8,  which  could  have  been  obtained  either  by 
counting  the  number  of  whole,  hexagonal  units  which  could  be  placed  in 
overlapping  fashion  in  the  figure  or  by  counting  the  number  of  interior  points. 
Others  seemed  to  count  all  the  points  (24).  Obviously,  these  students  did  not 
have  the  concept  of  area  as  a  covering  unit.  (See  Appendix  for  complete  list 
of  responses.) 

The  effect  of  partitioning  on  success  is  clearly  seen  at  Grade  12.  Both 
partitioning  and  success  rate  increased  at  the  older  level.  Over  half  of  the 
twelfth  graders  partitioned,  and  most  of  those  obtained  the  correct  answer. 
As  was  the  case  with  Grade  8,  most  of  the  students  who  did  not  partition 
counted  the  interior  points. 

Of  the  students  who  did  partition  but  obtained  an  incorrect  answer,  some 
counted  only  intact  hexagons,  not  recognizing  that  the  hexagons  themselves 
can  be  partitioned  into  congruent  triangles. 
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A  Question  of  Answers 


Eighth  Grade 


No  Partitioning,  (6.0%) 
Correct 


No  Partitioning,  (64.0%) 
Incorrect 


Partitioned,  (14.0%) 
Incorrect 


Partitioned,  (1 1 .0%) 
Correct 


(5.0%) 
Blank/Irrelevant 


Twelfth  Grade 


Partitioned,  (21.0%) 
Incorrect 


No  Partitioning,  (5.0%) 
Correct 


No  Partitioning,  Incorrect  (33.0%) 


Partitioned,  Correct  (32.0%) 


(9.0%) 
Blank/Irrelevant 


From  the  analysis  above,  it  appears  that  the  open-ended  item  gives  us  a  better 
indication  of  students'  conception  of  area  than  multiple-choice  items  in  which 
they  are  called  upon  only  to  supply  the  correct  answer.  In  these  cases, 
students  more  often  choose  the  perimeter  than  the  area. 

Consequently,  the  argument  is  made  that  students  difficulty  with  the  concept 
of  area  lies  in  vocabulary,  i.e.,  that  they  confuse  perimeter  with  area.  These 
results  suggest  a  more  fundamental  problem.  Students  do  not  understand 
area  in  terms  of  a  covering  unit,  nor  do  they  think  of  a  plane  figure  as  capable 
of  being  dissected  into  its  component  parts.  Their  difficulties  are  both 
conceptual  and  perceptual. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Measurement:   Precision  and  Accuracy 

Judgment  is  an  important  element  in  measuring.  In  addition  to  the  more 
mechanical  procedures,  students  have  to  be  able  to  decide  what  degree  of 
accuracy  is  necessary,  given  the  requirements  of  the  task. 


Grade  Four 

In  order  to  explore  such  questions  as  "Do  students  recognize  the  extent  to 
which  instruments  differ  in  precision?  Do  they  understand  the  relationship 
between  the  precision  of  the  instrument  and  the  task  involved?"  we  asked  the 
following  question  of  Grade  4  students. 


1 .     Christine  and  Carla  each  owned  a  chair  with  a  broken  leg.  They  decided  to  replace 
their  broken  legs.  They  both  measured  the  length  of  the  good  legs  on  their  chairs  to 
find  out  how  long  to  make  their  new  legs. 


1       23456789     10    11 


I'l'l'l'l'l'l'l'l'l 

23456789     10    11 


Christine's  Ruler  Carla's  Ruler 


Christine  measure  things  to  the  nearest  inch  using  her  ruler.  Carla  measures  things  to  the 
nearest  half  inch  using  her  ruler.  If  the  two  girls  decide  to  repair  chair  legs  as  a  hobby 
using  the  rulers  shown  above,  who  will  probably  be  more  successful?  Explain  why. 


Two-thirds  of  the  students  believed  that  Carla  would  produce  the  best  results. 
Most  recognized  the  need  for  precision  and  reasoned  that  her  measurement 
would  be  closer  to  the  actual  length  of  a  leg. 

I  think  Carla  would  be  more  successful.  I  think  so  because 
Christine's  ruler  is  by  the  inch,  and  Carla's  ruler  is  by  the  half.  So 
Carla's  ruler  would  be  closer  to  the  exact  measurement. 

Carla  will  be  more  successful  because  the  length  will  probably  not 
be  an  exact  inch  so  she  would  probably  have  about  the  right 
length  for  the  leg. 

Carla  would  because  if  the  chair  leg  was  131A  inches  long  312 
would  be  closer  than  13  or  14. 
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A  Question  of  Answers 


Twenty-nine  percent  believed  that  Christine  would  be  more  successful.  It 
was  clear  from  the  explanations  given  that  most  of  these  children  had  little 
understanding  of  the  relative  merits  of  precision  and  accuracy.  They  ap- 
peared to  look  at  the  task  and  reason  that  measuring  to  a  half-inch  was  more 
complex;  therefore,  more  difficult;  therefore,  there  would  be  less  chance  of 
accuracy.  This  confusion  may  have  come  from  their  own  experiences.  Some 
typical  responses  are  given  below: 

Christine,  because  it  will  take  her  less  time  and  work,  because  if 
you  measure  with  inches,  it  will  measure  easier. 

Christine  would  probably  be  more  successful.  People  usually 
measure  thing  by  the  nearest  inch  and  you  can  get  things  by  the 
inch  much  easier. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Numerical  and  Statistical  Concepts 


Estimation  and  Reasonableness 

The  importance  of  estimation  is  often  ignored.  We  may  think  of  it  merely  as 
a  short-cut  for  calculation  or  a  good  guess.  However,  students'  ability  to 
estimate  can  give  us  a  good  idea  of  how  well  they  understand  a  system  of 
measurement  (e.g.,  decimal,  linear,  volume)  and  what  they  consider  a 
reasonable  degree  of  accuracy. 


Grade  Four 

At  fourth  grade,  we  concentrated  on  numerical  estimation  in  the  following 
question: 


Explain  in  words  how  you  would  estimate  29  X  310.  Then  tell  what  your 
estimate  would  be. 


Sixty-five  percent  of  the  students  showed  an  understanding  of  what  is  meant 
by  estimation  in  this  context.  Approximately  half  rounded  29  to  30  and  310 
to  300,  while  another  15  percent  rounded  one  or  the  other  numbers  in  the 
pair. 
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A  Question  of  Answers 


Grades  Eight  and  Twelve 

In  these  grades,  we  included  estimation  tasks  in  both  numerical  and  verbal 
contexts.  The  numerical  question  was  similar  to  that  given  at  the  younger 
level  but  involved  decimal  fractions. 


Explain  in  words  how  you  would  estimate  88.74  X  105  in  your  head. 
Then  tell  what  your  estimate  would  be. 


In  this  case,  perhaps  the  simplest  method  would  be  to  round  105  to  100.  This 
was  the  course  followed  by  37  percent  of  eighth  graders,  with  26  percent 
allowing  for  some  sort  of  compensation  by  rounding  88.74  up  to  89  or  90. 
Others  rounded  105  up  to  110  (8  percent  of  students)  or  used  the  number  as 
given  and  rounded  88.74  to  a  whole  number  (19  percent).  However,  a 
sizeable  proportion  of  students  were  baffled  by  the  task.  Twenty  percent  left 
the  question  blank,  and  17  percent  completed  the  multiplication  with  the 
numbers  unchanged,  giving  a  description  of  their  multiplication  procedure. 
Although  a  few  rounded  their  final  answer  to  the  nearest  whole  number,  these 
students  obviously  did  not  understand  the  purpose  of  estimation. 

In  contrast  to  the  performance  of  fourth  graders,  the  results  of  these  older 
students  was  disappointing,  particularly  at  the  twelfth  grade.  Not  only  was 
there  little  growth  in  success,  but  it  could  be  argued  that  eighth  grade  students 
showed  a  better  understanding  of  the  task  than  the  older  students.  More 
twelfth  graders  appeared  to  perceive  a  need  for  an  absolutely  correct  answer 
rather  than  an  estimate. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Grades  Eight  and  Twelve 

Most  estimation  activities  take  place  in  a  context.  In  fact,  it  is  the  context  that 
determines  much  of  the  decision-making  in  estimation  — the  degree  of 
precision,  the  type  of  instrument  to  use.  The  practical  question  given  below 
deals  with  numerical  estimation,  but  puts  it  into  context. 


There  are  310  eighth  graders  at  Rockmont  Junior  High.  A  testing 
company  is  sending  310  tests  to  the  school  to  be  administered  to  the 
eighth  graders  in  their  homeroom  classrooms.  The  company  has  to 
estimate  how  many  direction  manuals  to  send.  There  should  be  one  for 
each  classroom.  Approximately  how  many  manuals  should  be  sent? 
Explain  how  you  came  up  with  your  estimate. 


Approximately  half  the  eighth  graders  and  70  percent  of  twelfth  graders 
replied  that  between  10  and  20  manuals  were  needed.  In  their  explanations, 
many  students  indicated  that  they  used  their  own  experiences  to  estimate  a 
reasonable  class  size.  However,  some  students  (15  percent  at  Grade  8  and  10 
percent  at  Grade  12)  were  puzzled  enough  by  the  question  to  leave  it  blank, 
while  5-7  percent  replied  that  the  necessary  information  was  missing.  A 
sizeable  proportion  of  eighth  grade  students  (20  percent)  focused  only  on  the 
numbers  themselves,  without  noting  what  they  represented. 

Some  typical  responses  were: 

39  manuals.  I  divided  8  into  310. 

5,  because  5  is  the  only  number  that  goes  into  310  evenly. 

38,  because  I  divided  310  by  8. 

This  type  of  response  — an  unthinking  manipulation  of  numbers  — is  also 
common  to  word  problems  on  multiple-choice  questions.  In  the  case  of  the 
open-ended  exercise,  the  requirement  that  they  explain  their  method  may 
have  alerted  them  to  examine  the  elements  of  the  problem  rather  than  to 
plunge  ahead  with  some  sort  of  calculation. 
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A  Question  of  Answers 


Statistical  Concepts:   Reasoning  about  an 
Average 

Although  multiple-choice  questions  can  be  used  to  measure  students'  ability 
to  evaluate  arguments  or  supply  missing  elements  or  even  identify  valid 
conclusions,  their  ability  to  generate  arguments  can  only  be  measured  by 
asking  them  to  do  it  —  either  orally  or  in  writing.  In  this  case,  we  asked  them 
to  construct  an  argument  concerning  the  effect  of  adding  a  constant  to  the 
numbers  in  a  set.  Because  of  its  familiarity,  we  used  the  concept  of  average 
test  score  to  determine  how  well  students  were  able  to  use  mathematical 
language  to  argue  convincingly. 


Roger  doesn't  believe  that  adding  a  constant  (the  same  number)  to 
every  student's  test  score  will  simply  change  the  average  test  score  by 
that  same  amount.  Use  the  space  below  to  convince  Roger  that  this  is 


It  was  obvious  that  students  were  unfamiliar  with  this  kind  of  question. 
Twenty  percent  of  eighth  graders  and  a  quarter  of  twelfth  graders  did  not  even 
attempt  to  answer  the  question.  Of  those  who  responded,  fewer  than  a 
quarter  of  the  students  at  Grade  8  were  able  to  give  a  convincing  argument 
using  a  worked  example  (i.e.,  compute  two  averages,  with  the  scores  in  the 
second  set  higher  by  a  constant).  At  Grade  12,  this  figure  rose  to  34  percent, 
while  another  4  percent  were  able  to  give  a  correct  narrative  explanation  for 
the  impact  of  a  constant  on  the  sum  of  the  scores.  Very  few  students  gave  an 
algebraic  proof  or  an  appropriate  logical  argument  in  mathematical  terms 
such  as  the  one  below  given  by  a  twelfth  grade  student.  (See  Appendix  for 
breakdown.) 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


Roger  doesn't  believe  that  adding  a  constant  (the  same  number)  to  every  student's  test 
score  will  simply  change  the  average  test  score  by  that  same  amount.  Use  the  space 
below  to  convince  Roger  that  this  is  true. 


7 


A  necessary  condition  for  a  convincing  argument  in  this  task  was  an  under- 
standing of  the  concept  of  average.  Here,  many  students  failed  before  they 
started.  For  example,  the  students  whose  work  are  shown  below  appeared  to 
interpret  "the  average  test  score"  as  the  relative  ranking  of  test  scores. 


Roger  doesn't  believe  that  adding  a  constant  (the  same  number)  to  every  student's  test 
score  will  simply  change  the  average  test  score  by  that  same  amount.  Use  the  space 
below  to  convince  Roger  that  this  is  true. 


r 
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A  Question  of  Answers 


Roger  doesn't  believe  that  adding  a  constant  (the  same  number)  to  every  student's  test 
score  will  simply  change  the  average  test  score  by  that  same  amount.  Use  the  space 
below  to  convince  Roger  that  this  is  true. 


^^bUJ± 


GO 


\<D 


_  Q7  >  ^  \  O         — 


Even  among  those  students  who  were  able  to  illustrate  the  effect  of  the 
constant,  few  recognized  that  only  two  numbers  are  needed  to  obtain  an 
average.  Most  used  four  or  more  in  each  set.  Furthermore,  approximately 
half  the  students  used  numbers  that  were  difficult  to  compute,  such  as  87, 79, 
and  93.  These  students  may  have  believed  that,  in  order  to  be  convincing,  the 
numbers  themselves  had  to  be  authentic.  They  did  not  understand  that  it  is 
the  quality  of  the  argument  rather  than  the  complexity  and  abundance  of  the 
numbers  themselves  that  is  critical. 


Roger  doesn't  believe  that  adding  a  constant  (the  same  number)  to  every  student's  test 
score  will  simply  change  the  average  test  score  by  that  same  amount.  Use  the  space 
below  to  convince  Roger  that  this  is  true. 


■kzJr   SCCKCC 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Math 


As  a  by-product  of  this  strategy,  10  percent  of  twelfth  graders  and  15  percent 
of  eighth  graders  were  unable  to  perform  the  correct  computation.  In  addi- 
tion, however,  there  appears  to  be  an  underlying  misconception  of  what 
qualifies  as  a  convincing  argument  in  mathematics.  Many  students  seemed 
to  believe  that  if  the  text  is  lucid  enough,  it  will  be  convincing.  Such  students 
personalized  their  responses,  restated  the  question,  or  relied  heavily  on  the 
use  of  the  vernacular  and  persuasive  language  in  their  presentation.  They 
were  distracted  by  the  context  and  seemed  unable  to  look  at  the  generalized 
mathematical  structure  involved. 

Although,  by  Grade  12,  a  few  more  students  were  correct,  more  used  easy 
numbers,  and  more  were  able  to  use  their  knowledge  of  algebra  to  make  a 
convincing  argument,  the  achievement  level  was  poor,  in  view  of  the  fact  that 
the  context  and  substance  of  the  question  should  be  familiar  to  all  students 
by  this  age.  The  majority  appeared  to  be  baffled  by  the  task. 
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Summary  of  Results 
and  Implications  for 
Instruction 


Variables  and  Relations:  Numerical  Patterns 

Finding  patterns  is  the  essence  of  mathematics.  It  is  disappointing,  there- 
fore, that  students  did  not  cope  more  adequately  with  the  tasks  in  this 
category  or  did  not  become  more  proficient  as  they  got  older.  On  the 
contrary,  fourth  graders  were  considerably  more  successful  with  this  type  of 
question  than  eighth  and  twelfth  graders.  The  results  suggest  that  this  kind 
of  task  may  be  more  unfamiliar  to  older  than  to  younger  students.  This  is  an 
ironic  conclusion,  given  the  dominance  of  algebra  in  the  high  school  cur- 
riculum. Certainly,  the  requirement  that  they  explain  their  reasoning  proved 
to  be  very  difficult  for  most  older  students.  However,  it  gave  us  valuable 
insight  into  students'  understanding  — insight  that  could  not  have  been  ob- 
tained if  we  had  been  satisfied  merely  with  a  correct  response.  From  their 
explanations  it  is  clear  that  many  students  arrived  at  what  seemed  to  be  a 
correct  solution  merely  by  chance  or  by  focusing  on  a  single  instance.  Most 
were  also  at  loss  to  give  an  equation  which  would  describe  the  functional 
relationship  between  the  two  sets  of  numbers.  Again,  generating  an  equation 
(as  opposed  to  recognizing  one)  proved  to  be  difficult  for  almost  all  eighth 
graders  and  for  nearly  three-quarters  of  twelfth  graders.  Even  those  who 
were  able  to  produce  the  correct  solution  could  not  look  beyond  the  specific 
case  to  give  a  generalized  rule  about  the  two  sets  of  numbers.  Formulating 
an  equation  demands  an  understanding  that  algebraic  equations  are  nothing 
more  than  a  symbolic  way  of  describing  an  existing  relationship  between 
numbers.  The  difficulty  that  students  had  suggests  that  teachers  make  a 
greater  attempt  to  provide  informal  activities  where  students  create  or  recog- 
nize patterns  and  generate  their  own  algebraic  sentences  to  describe  them. 
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Variables  and  Relationships:  Relationships  in 
Graphs 

Again,  fourth  graders  appeared  to  have  a  better  sense  of  what  graphs  are 
meant  to  portray  than  older  students.  It  may  be  that,  in  later  grades,  so 
much  emphasis  is  placed  on  the  mechanics  of  graphing  that  students  forget 
what  graphs  are  essentially  designed  to  portray  — a  relationship  between 
variables.  Whatever  the  cause,  a  disappointingly  small  number  of  eighth 
grade  and  twelfth  grade  students  were  able  to  interpret  or  draw  line  graphs 
that  described  relationships. 

The  argument  has  been  made  that  line  graphs  are  inherently  difficult,  relying 
upon  an  ability  to  abstract,  as  well  as  upon  proficiency  in  numerical  ability. 
On  the  contrary,  we  believe  that  line  graphs  are  too  useful  a  teaching  device 
to  wait  until  the  student  is  "ready."  This  type  of  graphing  should  not  be 
limited  to  those  students  who  have  shown  themselves  to  be  adept  at  numbers. 

Students  so  seldom  have  the  opportunity  in  mathematics  to  describe  relation- 
ships without  the  explicit  use  of  numbers  that,  when  asked  to  plot  a  linear 
equation,  they  often  don't  understand  what  they  are  representing.  They  don't 
realize  that  they  are  portraying  the  functional  relationship  between  variables. 
However,  the  problems  contained  here  allow  students  to  do  something  with 
quantities  other  than  compute. 

By  using  simple  graphs,  quickly  drawn,  without  scale  or  finely  determined 
coordinates,  students  can  focus  on  the  slopes  themselves  and  the  ideas  that 
they  are  meant  to  convey.  How  are  different  variables  related  to  one  another? 
How  often  are  they  linear?  Which  variables  have  similar  relationships? 
Which  have  inverse  relationships?  These  kinds  of  questions  can  make  math- 
ematics more  real  to  students,  as  well  as  developing  a  more  intuitive  under- 
standing of  later  work  in  algebra  and  calculus. 


Geometry  and  Measurement 

It  is  well  known  that  little  time  is  devoted  to  geometry  throughout  the 
grades.  It  is  not  surprising,  therefore,  that  students  don't  have  a  firm 
understanding  of  the  properties  of  a  figure  or  how  different  figures  relate  to 
one  another.  On  the  other  hand,  despite  the  fact  that  considerable  time  is 
spent  on  measurement  topics  such  as  area,  growth  in  understanding  is  slow 
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and  shaky.  Teachers  contend  that  the  problem  is  one  of  vocabulary  —  students 
confuse  the  words  perimeter  and  area.  Undoubtedly,  this  confusion  accounts 
for  some  errors,  but  responses  point  to  conceptual  and  perceptual  reasons  as 
well.  Students  do  not  understand  the  relationship  between  the  properties  of 
figures  and  their  measurement.  Perceptually,  they  fail  to  see  how  simple 
regular  figures  can  be  inscribed  in  curved  or  irregular  ones,  or  how  complex 
figures  can  be  partitioned  into  simpler  ones.  Conceptually,  they  don't  recog- 
nize that  area  is  the  measurement  of  surface  and  therefore  is  dependent  upon 
shape. 

The  most  prevalent  way  of  presenting  area,  through  regular  quadrilaterals, 
puts  too  much  and  too  early  a  focus  on  the  simplistic  application  of  formula 
to  the  detriment  of  understanding.  More  stress  on  irregular  figures  and 
non-standard  units  would  highlight  the  necessity  for  looking  at  the  figure  as 
enclosed  space.  A  simple  formula,  which  should  be  regarded  merely  as  a 
computational  short-cut,  too  often  becomes  the  only  tool  that  students  can 
use  to  solve  problems. 


Measurement:  Precision  and  Accuracy 

An  understanding  of  the  relative  need  for  precision  and  accuracy  is  also  a 
matter  of  judgment,  and  the  role  of  context  is  an  important  considera- 
tion. Approximately  two-thirds  of  the  fourth  grade  students  agreed  that  a 
more  precise  measurement  would  lead  to  better  results  when  attempting  to 
balance  chair  legs.  The  remainder  appeared  to  be  more  concerned  with 
accuracy,  believing  that  a  more  precise  measurement  would  be  more  difficult 
to  carry  out  and,  therefore,  stood  less  chance  of  being  "correct." 


Numerical  Concepts:  Estimation  and 
Reasonableness 

Estimation  is  not  the  same  as  "rounding  numbers,"  although  many  stu- 
dents appear  to  think  that  this  is  the  case.  Estimation  requires  both  a 
proficiency  with  a  system  of  measurement  (whether  it  be  decimal,  linear,  or 
some  other)  and  a  degree  of  common  sense,  what  some  would  call 
"reasonableness."  Approximately  65  percent  of  students  at  all  three  grade 
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levels  understood  the  meaning  of  the  word  and  were  proficient  enough  with 
the  decimal  system  to  successfully  carry  out  an  estimation. 

When  presented  with  a  problem  that  might  come  up  outside  the  classroom, 
eighth  graders  found  the  task  more  difficult.  Half  of  them  could  not  apply 
their  everyday  experience  (their  knowledge  of  the  number  of  students  in  a 
class)  to  this  "math"  problem,  and  many  retreated  to  a  stimulus-response 
reaction  of  manipulating  the  numbers  given.  Probably  reflecting  greater 
maturity,  more  twelfth  graders  (70  percent)  were  able  to  respond  reasonably, 
but  many  remained  baffled.  Their  answers  are  in  sharp  contrast  to  what  is 
known  about  the  "expert"  approach  to  problem  solving,  whereby  qualitative 
aspects  of  the  problem  are  of  primary  consideration,  to  be  followed  by 
numerical  calculations. 


Statistical  Concepts:  Reasoning  about  an 
Average 

It  is  important  to  reason  with  numbers,  such  as  in  the  practical  problems 
discussed  above.  It  is  also  important  that  students  be  able  to  reason  about 
numbers.  Unless  they  firmly  believe  that  mathematics  is  a  reasonable  system, 
students  will  be  reduced  to  accepting  and  memorizing  what  teacher  or 
textbook  says. 

What  was  designed  to  be  a  measure  of  reasoning  ability  acted  instead  as  a 
measure  of  students  understanding  of  the  concept  of  an  average.  It  was 
obvious  from  their  discussion  that  many  students  had  only  a  shaky  under- 
standing of  what  an  average  means.  Many  more  had  little  understanding  of 
the  hallmarks  of  a  mathematical  argument.  For  most  students,  a  convincing 
argument  was  one  that  was  specific  and  personal,  replete  with  realistic 
examples.  That  so  many  students  (from  one-fifth  to  one-quarter)  did  not  even 
attempt  the  question  suggests  that  they  have  had  little  experience  is  discussing 
either  mathematical  concepts  or  some  of  its  basic  applications.  Since  logical 
reasoning  is  the  hallmark  of  mathematics,  it  is  particularly  discouraging  that 
only  4  percent  of  twelfth  graders  could  give  either  a  correct  narrative  explana- 
tion or  algebraic  proof  of  an  average. 
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Conclusions 

The  questions  given  in  the  open-ended  section  of  the  assessment  yielded 
valuable  information  — not  only  about  what  students  know  and  under- 
stand, but  about  how  they  view  mathematics  itself.  That  so  many  students  did 
not  even  attempt  to  answer  the  questions  is  revealing.  Perhaps  they  did  not 
see  the  connection  between  the  questions  that  were  presented  and  what  they 
have  come  to  believe  mathematics  represents  — not  a  logical  system  of 
relationships,  but  computation.  In  this  case,  students  may  have  thought  that 
they  had  nothing  to  contribute.  When  asked  to  apply  basic  concepts  in  an 
unfamiliar  context,  they  were  baffled. 

However,  we  believe  that  the  value  of  this  type  of  testing  reaches  beyond  the 
specific  results  that  are  reported  here.  For  the  teacher,  open-ended  ques- 
tioning provides  a  better  insight  into  students'  understanding  than  those  more 
typical  questions  that  demand  only  a  correct  response.  Again  and  again  we 
see  that  an  answer  alone  is  but  a  weak  indicator  of  understanding.  In  contrast, 
when  we  ask  students  to  explain  or  to  justify  their  responses,  we  can  evaluate 
not  only  the  procedures  which  they  used,  but  the  premises  upon  which  those 
procedures  were  based.  Furthermore,  the  value  of  this  kind  of  task  extends 
beyond  the  context  of  assessment.  Any  one  of  the  exercises  given  in  this 
report  could  serve  as  launching  point  for  the  kind  of  discussion  that  educators 
say  is  so  important  — and  so  absent  — from  mathematics  instruction  today. 

Open-ended  questioning  is  valuable  for  the  student  as  well.  When  students 
are  asked  to  puzzle  and  explain,  to  apply  their  knowledge  in  an  unfamiliar 
context,  they  must  construct  meaning  for  themselves  by  relating  what  they 
know  to  the  problem  at  hand.  In  other  words,  they  must  act  like  mathe- 
maticians. This  kind  of  activity  encourages  them  in  the  belief  that  mathe- 
matics is  primarily  a  reasonable  enterprise,  founded  in  the  relationships 
apparent  in  everyday  life  and  accessible  to  all  students,  whatever  age  or  level 
of  ability.  It  is  not  by  accident  that  the  first  three  standards  put  forth  by  the 
National  Council  of  Teachers  of  Mathematics  focus  on  problem  solving, 
communication,  and  reasoning  in  the  study  and  teaching  of  mathematics. 
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Members  of  the  Mathematics  Advisory  Committee  reviewed  the  results  and 
made  the  following  comments. 


On  the  importance  of  graphical  exploration: 

It  is  apparent  that  students  at  all  three  grade  levels  did  not  use  par- 
titioning, diagrams,  pictures,  and  labels  to  help  them  understand  and 
solve  the  examples.  Some  of  the  students  who  did  carefully  erased  all 
their  work.  Teachers  should  put  more  emphasis  on  these  appropriate 
and  legitimate  techniques  for  problem-solving. 


Christine  Thompson 
Fourth  grade  teacher,  Leicester 


On  the  importance  of  assessing  students'  understanding: 

Much  more  can  be  determined  about  a  student's  understanding  of  a 
particular  concept  by  knowing  how  he/she  arrived  at  that  answer 
(whether  it  is  correct  or  incorrect).  This,  in  turn,  should  lead  to  better 
ways  of  teaching  these  concepts  to  children. 

It  may  force  us  to  examine  how  children  learn  (and  retain  what  they 
learn).  We,  as  teachers,  think  we  are  doing  such  a  great  job,  but  we 
are  not  always  aware  of  the  way  children  reason.  Are  we  responsible 
for  their  faulty  learning?  It  is  not  easy  for  a  child  to  explain  why  he/she 
did  something.  We  must  explain  things  to  them  in  a  meaningful  way.  I 
think  this  presents  a  good  challenge  for  us. 

Jack  Waite 

Director  of  Mathematics, 

Winchester 
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On  instruction: 


Everywhere,  it  seems,  the  open-ended  report  reveals  incomplete  math 
constructs  among  all  ability  levels.  I  believe  that  the  instruction  may 
be  incomplete  as  well  in  many  instances.  We  who  teach  math  are 
probably  not  taking  the  time  to  develop  sufficient  "enabling  skills"  for 
constructs  such  as  area  and  vector.  More  emphasis  is  needed,  then, 
on  enabling  activities  such  as  covering  (area),  what's  my  rule?  (func- 
tions/algebra), estimating  contests  (estimation), elementary  data 
analysis  (analysis  and  synthesis  skills  with  graphs),  and  better  develop- 
ment of  geometry. 

Thomas  Risoldi 
Director  of  Mathematics,  Salem 


On  the  importance  of  students'  explanations: 

The  most  important  finding  for  me  was  that  schools  do  not  ask  stu- 
dents to  justify  their  answers.  We  frequently  ask  for  an  answer,  but  sel- 
dom ask  why.  In  addition,  students  do  not  have  a  good  mathematical 
vocabulary.  They  do  not  know  how  to  "talk"  mathematics.  This  is  ob- 
vious when  they  try  to  explain  their  answers. 

Paul  Lyons 
Coordinator  of  Mathematics, 

Cambridge 


On  open-ended  questioning: 

The  open-ended  questions  in  the  statewide  tests  allow  the  committee 
to  observe,  and  thereby  report  to  other  educators,  the  thinking  proces- 
ses of  students  at  various  grade  levels. 

The  multiple-choice  type  questions  permit  only  an  interpretation  of 
resultant  answers,  but  not  thinking  patterns.  The  educational  implica- 
tions of  these  thinking  patterns,  presented  in  this  report,  must  be 
studied  carefully  by  all  people  involved  in  mathematics  education. 

Winston  Rose 

Department  Head, 

Masconomet  Regional 

High  School 
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Reporting  Category:     Patterns  and  Relationships:  Numerical 
Grade  Level:  Eight  and  Twelve 

Correct  number  solution: 


Grade  8             38  % 

Grade  12            55% 

Type  of  explanation  given  for  a  correct  response  (percentage  of  to 

correct 

correct 

incorrect 

incorrect 

other 

column 

function 

column 

function 

incorrect 

Grade  8              1 1 

10 

5 

3 

9 

Grade  12            17 

17 

5 

5 

11 

Correctness  of  equation  given  with  a  correct  response  (percentages  of  total 
sample) 


correct 

incorrect 

bis 

Grade  8 

13 

16 

2 

Grade  12 

28 

19 

8 
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Reporting  Category:     Patterns  and  Relationships:  Graphical 
Grade  Level:  Four 


Response  Summary  Frequency  (%) 

a    First  graph:  good  reason  66 

b.  First  graph:  poor  reason  14 

c.  Second  graph  1 5 

d.  Third  graph  4 

e.  Blank/ 1  don't  know  3 


Examples: 

a.  Graph  1  because  each  year  you  get  bigger.  In  the  other  ones  you  get  bigger  then  smaller. 


b.  Graph  1  because  Graph  2  and  3  are  uneven  and  wrong.  In  graph  2  it  shows  that  6  inches 
are  taller  than  12  inches.  In  graph  3  it  shows  that  1  inch  is  the  same  size  as  12  inches. 


c.  Because  some  years  you  grow  a  lot  more  than  other  years  when  you  grow  an  inch. 

c.  Because  some  1st  graders  are  bigger  than  third  graders.  Some  sixth  graders  are  smaller 
than  fourth  graders. 

c.  I  chose  graph  2  because  the  students  didn't  come  in  order  by  height,  they  took  it  the  way 
they  came. 


d.  Graph  3  because  some  kids  are  taller  than  other  kids  even  if  they're  in  a  lower  grade. 
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Graph  of  Study  Time  and  Grades 

Reporting  Category:     Patterns  and  Relationships:  Graphical 
Grade  Level:  Eight  and  Twelve 


Response  Summary 


Frequency  (%) 


Grade  8 


Grade  12 


a.  Complete  explanation  (accounts  for  both  24 
positive  and  negative  slopes) 

b.  Partial  explanation  (accounts  for  only  27 
positive  or  negative  slope) 

c.  Does  not  understand  meaning  of  graph  41 

d.  Blank/I  don't  know  8 


38 

31 

21 
9 


Examples  of  categories  (Grade  8): 

a.  You  need  to  spend  time  on  your  homework,  but  too  much  time  may  confuse  you.  That 
is  why  in  the  middle  it  is  the  highest  and  on  the  ends  it  is  the  lowest  on  the  grades  in 
math. 

a.  People  who  didn't  study  very  much  or  not  at  all  didn't  do  very  well.  People  who  studied 
until  they  knew  it  got  the  top  scores.  People  who  studied  until  they  were  exhausted  or 
too  much  didn't  do  as  well  because  they  could  have  confused  things. 


b.  Because  the  less  time  they  spent  on  homework  each  evening,  the  lower  the  grade.  The 
more  time  they  spent,  the  higher  the  grade. 


c.  It's  just  the  way  that  it  came  out  after  he  connected  the  dots  that  showed  these  test  grades 
according  to  the  average  time  they  spent  on  homework. 

c.  The  graph  has  this  shape  because  in  the  beginning  the  students  did  less  homework,  so 
their  grades  were  poor.  Then  they  studied  more  and  got  much  higher  grades,  until  they 
thought  their  grades  were  good  enough.  They  didn't  have  to  study  much  anymore  and 
their  grades  went  down  once  again. 

c.   It  is  that  way  because  on  one  night  that  student  had  studied  more  than  other  nights. 
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Time/Age  Graphs 

Reporting  Category: 
Grade  Level: 


Response  Summary 


Patterns  and  Relationships:  Graphical 
Eight  and  Twelve 


a.  Acknowledges  changing  pattern 

b.  Negative  slope:  good  explanation 

c.  Positive  slope:  good  explanation 

d.  Explanation  differs  from  graph  drawn 

e.  Other  incorrect  or  irrelevant  slopes 
and  descriptions 


f.    Blank/ 1  don't  know 


Examples  (Grade  8) 


a. 


Time    Needed 

to    Run    Mile 

(minutes) 


Age    (years 


Frequency  (%) 


Grade  8 


M      F 


Grade  12 


M      F 


18 

15 

33 

23 

15 

10 

5 

9 

16 

14 

11 

31 

26 

22 

35 

16 

10 

26 

4 

6 

13     12 


13     15 


When  you  are  young  you  don't  go  as 
fast,  but  when  you  get  older  you  get 
tired  easily. 
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b. 


Time    Needed 

to    Run    Mile 

(minutes) 


The  older  the  age  was,  the  faster  the 
person  would  run. 


Age    (years) 


Time    Needed 

to    Run    Mile 

(minutes) 


The  older  you  get  the  more  time  you 
need  to  run. 


Age    (years) 


d. 


Time    Needed 

to    Run    Mile 

(minutes) 


When  you  are  real  young  you  can't  run 
that  fast,  but  as  you  get  older,  you  get 
better.  But  after  awhile  your  speed 
slows  down  as  you  go  up  in  age. 


Age    (years) 


e. 


Time    Needed 

to    Run    Mile 

(minutes) 


The  older  you  are,  you  need  to  run  more 
to  make  you  healthy. 


Age    (years) 
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e. 


Time    Needed 

to    Run    Mile 

(minutes) 


Age    (years) 


It  would  depend  on  the  age  and  if  you 
were  in  shape  to  do  the  mile. 
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Reporting  Category:    Geometry  and  Measurement:  Plane  Figures 
Grade  Level:  Four 

Response  Summary  Frequency  (%) 

a.  Correct,  complete,  defining  description  3 

of  rectangle 

b.*  Correct,  informal  description  of  rectangle  9 

(e.g.,  "corners"  for  angles) 

c.  4  sides  and  pairs  of  equal  sides  (no  25 
mention  of  angles) 

d.  Pairs  of  equal  sides,  horizontal  longer  25 
than  vertical 

e.  Other  description  8 

f.  Refers  to  appropriate  real  objects  (e.g.,  5 
dollar  bill) 

g.  Refers  to  rectangular  solid  3 
h.  Describes  triangle  13 
i.    Blank,  irrelevant  9 


Definitions  categorized  as  correct  do  not  have  to  describe  all  charac- 
teristics of  a  rectangle.  For  example,  "4  sides,  opposite  sides  parallel,  one 
right  angle"  must  be  a  rectangle  even  though  no  mention  is  made  of  the 
opposite  sides  being  equal  in  length.  All  figures  described  are  assumed  to 
be  closed. 
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Estimating  Need  for  Manuals 

Reporting  Category:     Numerical  Concepts:  Estimation  and 

Reasonableness 
Grade  Level:  Eight  and  Twelve 


Response  Summary 


a.  10-20  manuals 

b.  Explanation  describes  appropriate 
method:  calculation  error 

c.  "Not  enough  information" 

d.  Other 

e.  Blank/ 1  don't  know 


Frequency  (%) 


Grade  8 


54 


Grade  12 


71 


20 


15 


10 


Examples  (Grade  8): 

a.  12.  You  say  about  25  in  each  classroom,  so  you  divide  310  by  25  and  whatever  your  answer 
is,  that  is  how  many  booklets. 

a.  15.  There  is  probably  about  20  students  in  each  class.  Divide  20  into  how  many  students. 
This  will  give  you  how  many  homerooms  there  are. 


b.  Approximately  104  because  there  are  approximately  30  students  in  each  class,  so  you 
divide  310  by  30  and  round  off,  just  in  case  there  are  more  students  per  classroom. 
(Student  did  not  recognize  need  for  decimal  in  division.) 


c.    You  have  to  know  how  many  homerooms  there  are. 


d.   Approximately  5.  There  are  two  junior  high  grades.  So  divide,  then  estimate  that  there 
are  about  31  kids  in  a  class.  (310/2  =  155. 150/30  =  5) 

d.  There  are  about  ten  students  per  homeroom.   So  therefore  you  would  have  to  send 
around  31  manuals. 
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d.   5.  If  there  were  62  students  in  one  classroom  then  5  manuals  were  needed. 

d.   74  test  manuals.  Divide  310  by  4  homerooms. 

d.    I  would  assume  4  because  that  is  the  number  of  majors  (math,  English,  science,  social 
studies)  and  homerooms  are  usually  located  in  the  4  major  subject  classrooms. 
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Reasoning  about  an  Average 

Reporting  Category:     Statistical  Concepts 
Grade  Level:  Eight  and  Twelve 


Response  Summary 


a.  Correct  narrative  explanation  or 
algebraic  proof 

b.  Computes  2  averages,  second  is 
higher  by  a  constant 

c.  Adds  constant  to  score,  does  not 
compute 


Frequency  (%) 


Grade  8 


0 


22 


Grade  12 


34 


d 

Computational  error  in  attempt  to 
average 

9 

6 

e. 

Incorrect  or  unconvincing  narrative 

18 

15 

f. 

Irrelevant,  misunderstands  question 

26 

15 

q. 

Blank 

20 

24 
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790  Turnpike  St. 

North  Andover,  MA  01845 

508/689-0164  or  617/727-0600 


Laurie  Slobody 

Central  Mass.  Regional  Center 

Beaman  Street,  Route  140 

West  Boylston,  MA  01583 

508/835-6266 

Peter  Cirioni 

Greater  Boston  Regional  Center 

75  Acton  Street 

Arlington,  MA  02174 

617/641-4870 

Pat  O'Brien 

Southeast  Regional  Center 

P.O.  Box  29 

Middleboro,  MA  02346 

508/947-1231 


Related  publications  about  the  Massachusetts  Educational  Assessment  Program: 

•  The  Massachusetts  Educational  Assessment  Program:  1988  Statewide  Summary 

•  Using  the  School  Report 

•  Reading  and  Thinking:  A  New  Framework  for  Comprehension 

•  Science  in  the  Elementary  Schools 

•  Moving  Geometry  from  the  Back  of  the  Book:  A  Report  on  Geometry  and 
Measurement  in  the  1986  Assessment 

•  Social  Studies  Report 

•  On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Reading 

•  On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Mathematics 

•  On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Science 

•  On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Social  Studies 
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